First Generation artificial neural networks were based on the simplified neural model of Warren S. McCulloch and Walter Pitts. The McCulloch-Pitts neuron was presented in their 1943 paper “A Logical Calculus of Ideas Immanent in Nervous Activity”. The McCulloch-Pitts neuron is also known as a Threshold Gate, which takes a plenitude of Boolean inputs and returns a single Boolean output. The output is logic ‘1’ when the inputs are greater or equal to a defined threshold value. The transfer function is a logic AND, OR or NOT function. First generation neural networks used the McCulloch-Pitts neuron as the basic computation unit in a single layer without feedback.
Second generation artificial neural networks are based on McCulloch-Pitts neurons modified to use a sigmoid activations function and a continuous set of possible output values. In 1957 the ‘Perceptron’, also known as the MARK1 was presented at the Cornell Aeronautical Laboratory, in a paper by Frank Rosenblatt. The Perceptron is a single-layer, feed-forward artificial neural network.
Third generation artificial neural networks are based on ‘integrate and fire’ neurons, whereby the synaptic strength is expressed as a static value. Such networks are trained by manually or programmatically adjusting this static value. Most neural network models are based on the following three assumptions. Firstly, the efficacy of a synapse in generating a synaptic potential is assumed to be static for a resulting action potential in neurons. The efficacy of a synapse is essentially a constant. Certain models modify this assumption by allowing a slow variation over a period of processing many variables. In the second assumption, each sending neuron provides the same signal to all other neurons to which it is connected by some means. Thirdly, the network is trained by direct or indirect manual means. Most networks are feed-forward networks with no feedback.
A common artificial neural network used in predictive and analysis machines is the Hopfield network. Nodes in a Hopfield network are static binary threshold units. The output Alpha_i of a unit can either be logic 1 or logic 0, if the summed input exceeds the threshold value Phi: E represents the energy of the junction. Wij is the strength of the connection. S is the state of unit j and Phi is the threshold value. A Hopfield network stabilizes at the minimum energy level at all junctions. Boltzmann machines add an annealing factor to the Hopfield equation. Boltzmann machines are capable of learning limited internal representations.
In previous instances of neural networks many of the neuron functions have been compromised in order to force functional results. This compromise has resulted in dedicated machines while the biological model is in contrast adaptive. The mentioned networks are based upon antiquated models of biological neurons whereby the temporal character of activation patterns and the functions of feedback and inhibition are largely ignored. The model that is presented here removes these assumptions allowing the construction of adaptive autonomous learning neural networks.
Function libraries have been used in computer programs for some time. Dynamic Link libraries are extensive used in computer programs today. A Dynamic Link Library provides external functionality to computer programs through the substitution of call addresses. In addition to Dynamic Link Libraries, programming libraries provide source code or machine code that the programmer can include in programs. In such cases the functions are called directly and are included in the object code when the program is compiled. Specific programming libraries for Artificial Intelligence applications contain functions, expressed as programming steps, which control certain aspects of the Artificial Intelligence procedure. Each Artificial Intelligence application program is individually coded and no growth path or re-usable code is generated. In learning systems, the learning function is coded as programming steps and limited to a narrow scope within the range of the application program. In contrast, the functions in a Dynamic Neural Function Library are not called from programs and do not comprise program steps. The functions in the Dynamic Neural Function Library are expressed as values which represent the properties of temporal-spatial patterns, which represent a function when they are uploaded or combined in an Intelligent Target System. A common hardware platform, specifically designed for the creation of cognitive systems, aids in the creation of a generic growth path. Dynamic Neural Function Libraries complete the creation of a growth path with re-usable and combinable functions.